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ABSTRACT 

Motivating students to come prepared to class is often a futile exercise. The consequences are magnified as more 
instructors adopt the flipped classroom teaching model, in which students are expected to come to class with some 
knowledge of the material This paper analyzed the implementation of a pedagogical tool that addresses this 
problem: multimedia pre-lectures. We tested the efficacy of pre-lectures on students with no prior exposure to 
college or AP-level economics using a controlled clinical experiment, focusing on introductory economics concepts. 
Students who had access to pre-lecture materials scored significantly higher on comprehension and retention than 
those with access only to textbooks. 
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INTRODUCTION 


scent trends in higher education have focused on new pedagogical techniques to increase student 
\Ty/\ motivation and comprehension. This is important because the way in which students learn has changed 

U as a result of new technology in media and communications. Students today tend to be more visual 

learners that acquire information through short bursts of content, such as those on social media sites. Moreover, the 
growth of online resources and technologies that complement, and in many situations, substitute, for the traditional 
“chalk-and-talk” lectures has made pedagogy an important issue. Instructors today have access to a multitude of 
teaching tools as they are increasingly being offered by publishers as part of their suite of learning resources. 


One trend in higher education is the flipped classroom, in which students are asked to review classroom material 
prior to class, allowing more class time to be used for active learning activities such as group discussion, problem 
solving, and experiments. The flipped classroom model takes much of the traditional lecture outside of the 
classroom (becoming the “homework”), while activities previously completed outside of class, such as problem 
solving and peer-to-peer learning, are moved into the classroom. To facilitate the learning of essential course 
concepts prior to class, instructors typically assign some form of pre-class assignment using various learning tools. 
This can be as simple as a set of textbook readings, or it can be a multimedia tool such as PowerPoint slides or 
videos to serve as the pre-lecture. 


This paper examined whether multimedia pre-lectures are more effective than traditional textbooks in preparing 
students for an economics principles class based on three objectives: (a) motivating students to study course material 
before class, (b) conveying key concepts in a way that students can comprehend when seen for the first time, and (c) 
increasing long-term retention of the concepts. We used a controlled clinical trial to evaluate the effectiveness of 
text-based readings and multimedia pre-lectures. 

In the clinical trials, students with no prior exposure to college or AP-level economics were randomly assigned into 
one of two groups: one that was given only readings from a well-known textbook, and another that was given 

Copyright by author(s); CC-BY 37 The Clute Institute 



Journal of College Teaching & Learning - Second Quarter 2016 


Volume 13, Number 2 

multimedia pre-lectures covering the same topics. Students in both groups were then given the same assessment 
immediately after the session as well as 2 weeks later to evaluate longer-term retention. Based on the clinical trials, 
we found evidence that supports the conclusion that multimedia pre-lectures enhanced both the comprehension and 
retention of economics concepts. Students who watched multimedia pre-lectures scored almost 6 points higher in the 
immediate assessment than those who had access only to a textbook, and almost 8 points higher in an assessment 
given 2 weeks later. 

In the remainder of this article, the case for multimedia presentations is presented by a summary of research on the 
effectiveness of visuals in learning outcomes. The clinical trial study and the methodology used is then described, 
followed by a discussion of the results and their implications. Finally, the importance of our findings is summarized 
along with direction for further research. 

THE CASE FOR VISUAL PRESENTATIONS 

The debate regarding the effectiveness of text versus video is one that stirs strong feelings in the academic 
community. Purists believe that online technologies and multimedia tools pose a distraction to learning, with the 
argument that the best form of teaching requires only a well-prepared instructor and a chalkboard, complemented by 
a textbook. But a growing number of instructors have adapted to technology, complementing their classes with a 
variety of tools. They argue that today’s college students are part of a new generation that grew up with mobile 
devices and the Internet, and therefore are more accustomed to learning using these tools. 

The case for using visual presentations in lieu of text-based presentations in the teaching of economics focuses on 
two general areas of research: (a) research in neuroscience and cognitive science that studies how the brain functions 
when stimulated by information presented in various forms, and (b) studies of how existing forms of visual 
presentations within the economics discipline affect the learning outcomes of students. For the latter, we focused our 
analysis on economics principles, with an emphasis on assessment outcomes and content retention. 

Neuroscience and Cognitive Science 

Although any presentation, including written words or bullet points on a slide, is inherently visual because one must 
use eyes to see it, for purposes of our research we defined visuals as the use of photos, diagrams, animations, and 
videos instead of words to deliver an educational message. The rationale for visual presentations derives from 
decades of research in both neuroscience and cognitive science with respect to two areas of brain function: vision 
and memory. An in-depth discussion of this research is found in Vazquez and Chiang (2014), which we briefly 
summarize in the following paragraphs. 

The first biological argument for visual presentations is that we see in pictures. One reason why text is less capable 
than pictures to deliver any message is because the brain sees words as lots of tiny pictures. Data show that a word is 
unreadable unless the brain can separately identify simple features in the letters (Pelli, Farell, & Moore, 2003). 
Hence, reading creates a bottleneck. To our cortex, there is no such thing as words. In fact, when people read, they 
tend to create mental images of what the text is conveying. 

Therefore, the more visual the input becomes, the more likely it is to be recognized and recalled, an effect referred to 
as the pictorial superiority effect (Endestad, Magnussen, & Helstrup, 2003; McBride & Dosher, 2002; Stenberg, 
2006). Specifically, the process of vision takes place not in the eye, but in the brain which reconstructs information 
received from the eyes with previous knowledge stored in memory (Carlson, 2007; Medina, 2008; Fried, Munch, & 
Werblin, 2005; Roska, Molnar, & Werblin, 2006). These studies help to justify why the use of visuals enhances the 
way knowledge is conveyed in the classroom. 

The other argument for visual presentations, the dual coding theory, comes from cognitive science, and suggests that 
people receive and process information through two distinct but interrelated channels: a verbal channel, which 
processes verbal material, and a visual channel, which processes pictorial material and nonverbal sounds (Paivio, 
1986; Paivio, 2006). 


Copyright by author(s); CC-BY 


38 


The Clute Institute 



Journal of College Teaching & Learning - Second Quarter 2016 


Volume 13, Number 2 

Figure 1. Memory and the cognitive case for adding pictures to educational lectures 



Dual-coding theory leads to two implications for classroom instruction: first, adding an image to any verbal message 
can enhance the retention of that message in one’s memory, and second, a message that is solely text-based has the 
potential to overwhelm the verbal channel, making retention more difficult. Moreover, when one merely narrates 
written words, thereby emphasizing words though both the auditory and visual sensors, the ability to process the 
message may be inhibited due to the verbal channel being overloaded. 

Effect of Visuals on Learning Outcomes 

Much of the recent literature on the role of technology in the classroom has focused on the effect that online courses 
have in supplanting face-to-face courses at universities throughout the United States. Because online courses are 
natural laboratories for testing the effectiveness of technologies such as learning management systems, lecture 
capture, video clips, and multimedia slides, much of the existing literature has focused on how the movement toward 
online courses has affected learning outcomes in economics courses. Because a flipped classroom relies on moving 
the traditional lecture online into a pre-lecture format, it is useful to briefly examine the effectiveness of online 
delivery of courses in general. 

Recent studies have suggested that although online courses may not be a perfect substitute for a face-to-face 
classroom, the differences in student outcomes are not significant, and tend to impact certain subgroups of students 
more than others. For example, Figlio, Rush, and Yin (2013) found that the performance in face-to-face economics 
courses is marginally higher for Hispanic students, male students, and lower-performing students. Subsequent 
studies in economics and statistics by Joyce, Crockett, Jaeger, Altindag, and O’Connell (2014) and by Bowen, 
Chingos, Lack, and Nygren (2014), respectively, showed that courses that met in person more often performed only 
slightly better than hybrid courses. Bosshardt and Chiang (2016) focused on selection issues in online economics 
courses. When students have a choice between online and face-to-face formats, that choice is influenced mostly by 
personal attitudes toward technology and previous experience with online courses. They conclude that if chalk-and- 
talk can be replicated online, then the advantage of a live classroom lies in the ability to engage in active learning. 

A recent compendium of 80 topics on economics education, edited by Hoyt and McGoldrick (2012), contains 
contributions from dozens of instructors and researchers aimed at improving learning outcomes and concept 
retention. Although active learning and flipped classrooms were among the more prominent topics discussed, much 
of the existing research on the effectiveness of a flipped classroom framework has involved other disciplines, most 
notably physics, which was arguably the pioneer of the flipped classroom (Stelzer, Gladding, Mestre, & Brookes, 
2009). Today, there are an increasing number of papers and books discussing best practices on flipping a classroom 
in many disciplines (e.g., Bergmann & Sams, 2012; Roach, 2014; Vazquez & Chiang, 2015). 

The primary challenge of implementing a flipped classroom is determining the best method of conveying essential 
concepts to students before the class meeting. Historically, this has been encouraged by way of reading the 
corresponding textbook chapters prior to class. Although instructors have long encouraged students to engage in 
such proactive learning, empirical and anecdotal evidence suggest that most students do not read textbooks prior to 
class. For example, in an informal poll of the author’s large principles of economics classes in the Fall of 2013, 71% 
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of respondents indicated that they “never” or “rarely” read the assigned textbook readings prior to class, despite the 
fact that extra credit clicker questions were posed during class as an incentive to students to read the textbook. 

The solution that has become increasingly popular is the development of mini-lectures that consist of either narrated 
animations or live videos. These short lectures are typically 10 to 15 minutes in length, visually-intensive, and often 
are embedded with questions to assess student understanding and retention. The result has been the creation of two 
types of mini-lectures: (a) pre-lectures, which assume that students have no prior knowledge of the topic and are 
aimed at students prior to class; and (b) summary lectures, which serve as a review tool for students to watch after a 
class. Pre-lectures tend to be used more frequently in a flipped classroom setting, and is the format we studied in this 
research. 

Studies of the effectiveness of pre-lecture videos on learning outcomes using controlled clinical trials have rarely 
been conducted. Instead, most studies analyze outcomes using an instructor’s own classes. However, that approach 
is subjected to the biases stemming from external factors influencing a student’s performance in a class that a 
clinical trial in a controlled lab setting can avoid. Prior clinical trials focusing on the effectiveness of videos include 
those by Stelzer et al. (2009), who found a profound positive effect on learning outcomes in physics, and Lusardi, 
Samek, Kapteyn, Glinert, Hung, and Heinberg (2014), who concluded that videos were much more effective than 
written narratives in conveying financial literacy. Both studies were conducted in a lab setting, and both showed that 
the use of visual tools and video is more engaging and results in greater confidence in knowledge which translates to 
improved learning. 

This article focuses on two learning outcomes: (a) the extent to which students without any prior knowledge of a 
topic is able to recall and understand economics concepts, and (b) the extent to which students are able to retain 
information over a longer term. Both learning outcomes are critical for assessing the level of preparedness by 
students prior to class. These results carry important implications for the future of teaching pedagogy such as the 
flipped classroom and the resources that are being developed to prepare students for class. 

EMPIRICAL STUDY DESCRIPTION 

In order to test the benefits of multimedia pre-lectures, we focused on four major topics in a principles of 
microeconomics: production possibilities frontier (PPF), the benefits of trade (Trade), price elasticity of demand 
(PED), and other elasticities (OE). These are typically among the most challenging topics in an economics principles 
course, and therefore are ideal to investigate which type of presentation mode works best to help students learn and 
retain economics concepts. 

Multimedia Pre-Lectures 

We developed four multimedia learning modules (pre-lectures), one for each of the four topics addressed in this 
research. Each pre-lecture covered one lecture’s worth of course content and was divided into approximately eight 
scenes. Each scene was implemented as a video containing dynamic animations synchronized with an audio 
narration that was controlled by the student (pause, play, and rewind). We also included embedded formative 
questions in two or three scenes in each pre-lecture. These questions must be answered correctly before the student 
can proceed to the next scene. Note that the pre-lectures were not designed to replicate an entire lecture, but instead 
to provide a visual introduction to the key concepts. The average narration time of a pre-lecture is 12 minutes, but 
students took an average of 17 minutes to complete a single pre-lecture including the embedded questions. 

The second presentation mode consisted of a brief set of pdf pages from a leading principles textbook: Mankiw’s 
Principles of Microeconomics 6 th edition (2011). We used a few pages of text from each topic along with the worked 
examples and practice problems. The use of these pages for research purposes falls within fair use guidelines; the 
total number of pages used constituted less than 4% of the book. The length of the readings was limited to an 
amount that could adequately be read and studied within the allotted time limit. 
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Study Procedures 

We recruited volunteers during the Fall 2013 semester by offering an incentive of a $50 Amazon gift card for 
participating in the study. All participants had never previously taken a college or AP-level economics course. All 
students came to a computer lab for three different sessions: two base sessions and a retention session. Each base 
session consisted of two parts: presentation and assessment, each lasting about 30 minutes. In the presentation phase, 
students were randomly assigned into one of two groups: study and control. 

• Study group (Video): Students in this group were shown the multimedia pre-lecture videos. 

• Control group (Text): Students in this group were shown the textbook chapters. 

Table 1 shows descriptive statistics for relevant study variables between the control group and study group. As the 
statistics show, there are no significant differences between the two groups for any variables, which suggests that the 
process of randomization of students into the samples was successful. 


Table 1. Descriptive Statistics for Relevant Study Variables 



N 

Minimum 

Maximum 

Mean 

Std. Deviation 

Group 

Group 

Group 

Group 

Group 

Text 

Video 

Text 

Video 

Text 

Video 

Text 

Video 

Text 

Video 

Male 

41.0 

41.0 

0.0 

0.0 

1.0 

1.0 

0.6 

0.5 

0.5 

0.5 

Age 

41.0 

41.0 

19.0 

19.0 

30.0 

26.0 

21.2 

20.7 

2.3 

1.5 

ACT Composite 

37.0 

39.0 

21.0 

20.0 

35.0 

35.0 

29.4 

29.3 

4.1 

3.9 

Cumulative GPA 

41.0 

41.0 

1.5 

0.7 

4.0 

4.0 

3.2 

3.2 

0.6 

0.8 

Math Course Credit Earned 

41.0 

41.0 

0.0 

0.0 

19.0 

18.0 

4.8 

5.4 

6.0 

5.4 


The assessment phase consisted of a quiz with 24 multiple choice questions relating to the material covered in the 
presentation phase. All students returned to the lab for a retention session 2 weeks after the last base session to 
complete a quiz consisting of 20 multiple choice questions. This session lasted 30 minutes. 


Figure 2. Experimental design at a glance. 

Base Session 1 (Monday) 

Base Session 2 (Wednesday) 

Retention Session (2 weeks later) 

• PPF and Trade topics 

• Presentation (30 minutes): Students 
watch pre-lecture videos or read booklet 

• Assessment (30 minutes): 24 multiple 
choice questions 

• PED and OE topics 

• Presentation (30 minutes): Students 
watch pre-lecture videos or read booklet 

• Assessment (30 minutes): 24 multiple 
choice questions 

• Assessment (30 minutes): 

Students take an assessment 
exam consisting of 20 multiple 
choice questions. 


METHODOLOGY 

Our first hypothesis was that students in the study (video) group would achieve a higher assessment score than those 
in the control (textbook) group. To test this hypothesis, we used the following regression equation: 

Model 1: Post-Assessment Score = fio + fiivideoi + fcACTi + p$Xi + e z - 

where Post-Assessment Score is the score the student obtained in an assessment (consisting of 48 multiple choice 
questions) taken immediately after the study session (watching the videos or reading the text); video is a dummy 
variable equal to 1 if the student was in the study (video) group, ACT is the score that the students achieved on the 
standardized ACT exam as reported from their transcripts, and A is a set of demographic variables, including age 
and gender. The parameter of interest is which represents the change in the average score resulting from 
watching the pre-lectures in lieu of reading the text. 
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Our second hypothesis is that presenting the material in video format will also improve a student’s retention of the 
concepts. To test this hypothesis, we used the following regression equation: 

Model 2: Retention Score = fio + fiivideoi + fcACTi + /?AG + Si 

where Retention Score is the score the student achieved from an assessment consisting of 20 multiple choice 
questions taken 2 weeks after the initial assessment taken immediately after the study session. Students from both 
groups took the same assessment quiz for the purposes of evaluating the extent of subject matter retention. 

EMPIRICAL RESULTS 

Table 2 shows the ordinary least squares (OLS) regression results from Model 1. Holding all other variables 
constant, students in the study (video) group on average scored almost 6 points higher on the assessment quiz than 
students in the control (textbook) group. These results are statistically significant at the 0.05 level. Therefore, our 
hypothesis that students are able to learn and understand the material better by watching videos, instead of reading 
text, is supported by our results. 


Table 2. Results from OLS Regression on Model 1 (n = 82) 


Model 

Unstandardized Coefficients 

B 

Std. Error 

Group 

5.825* 

2.768 

Gender 

4.891 

2.996 

Age 

2.559* 

1.201 

Math Course Credit Earned 

0.463 

0.273 

ACT Composite 

2.201** 

0.392 


Notes: ** p < 0.01, * p < 0.05 


Table 3 shows the OLS regression results from Model 2, estimating the retention of concepts 2 weeks later. Holding 
all other variables constant, the study (video) group scored almost 8 points higher in the retention assessment quiz 
than a student in the control (textbook) group. These statistically significant results suggest that the effect of 
multimedia videos had an even greater benefit on concept retention. 


Table 3. Results from OLS Regression on Model 2 (n = 82) 


Model 

Unstandardized Coefficients 

B 

Std. Error 

Group 

7.758* 

3.340 

Gender 

4.255 

3.614 

Age 

1.863 

1.449 

Math Course Credit Earned 

0.533 

0.329 

ACT Composite 

1.921** 

0.473 


Notes: ** p < 0.01, * p < 0.05 


Both assessment test scores, base and retention, included all questions from the assessments. However, some 
questions are different in terms of the skills required to answer them. To evaluate whether the marginal benefit of 
multimedia presentations is different depending on the type of question asked, we classified all questions according 
to Anderson and Krathwohl’s updated version of Bloom’s taxonomy of cognition domain (Anderson & Krathwohl, 
2001). Level-1 includes questions requiring remembering, Level-2 includes questions requiring understanding, 
Level-3 includes questions requiring applying, and Level-4 includes questions requiring analyzing. We ran the 
model again for each of the four categories. Table 4 shows the results of the OLS regression coefficients for these 
regressions. 
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Table 4. OLS Estimates By Cognitive Domain Level (n = 82) 



Level-1 

Level-2 

Level-3 

Level-4 

Group 

5.73 

9.06** 

1.60 

16.58* 

(3.81) 

(3.60) 

(3.53) 

(7.71) 

Gender 

3.97 

7.72* 

0.70 

8.10 

(4.13) 

(3.90) 

(3.87) 

(8.49) 

Age 

3.94* 

1.60 

2.85* 

-0.42 

(1.66) 

(1.57) 

(1.54) 

(3.38) 


0.41 

0.28 

0.46 

1.70* 

Math Course Credit Earned 

(0 ' 38 L 

(0.37) 

(0 - 37 L 

(° J6 l 

ACT Composite 

2.30 

1.72*“ 

2.64 

2.10 

(0.54) 

(0.51) 

(0.50) 

(1.11) 

Adjusted R-squared 

0.28 

0.26 

0.34 

0.20 


Notes: Standard errors in parentheses. ***/? < 0.01, ** p < 0.05, * p < 0.1 


The results suggest that the marginal benefit from multimedia lectures was highest, and most statistically significant, 
for levels 2 and 4; understanding and analyzing, respectively. In fact, students in the video group scored 16 points 
higher on questions in level 4. These results suggest that the benefits of multimedia pre-lectures depend on the type 
of question, particularly its cognitive level. 


To further contrast the effectiveness of multimedia pre-lectures and textbook readings, we asked each participant to 
convey their opinions on their experience with the learning process. Specifically, we asked how confident they felt 
with their understanding of the material after finishing the module. The results are shown in Figure 3. Of the 
students in the Video group, 44% felt they understood the concept either well or very well. This is in comparison to 
just 36% in the Text group. Therefore, the results suggest that students in the Video group not only scored better, but 
also self-reported better understanding of the material. 


Figure 3. Responses to “After completing this module, how well did you understand the concept being presented?” 



TEXT 
■ VIDEO 


Our findings from the clinical trials point to several important takeaways. First, from a cognitive and neurological 
perspective, individuals comprehend and retain more information when absorbed using both the visual and verbal 
channels. In our case, a visual and audio presentation performed better than a text-based presentation. Second, from 
a practical perspective, nearly all students today grew up with Internet and mobile technologies, and therefore are 
generally more comfortable learning using these tools. Our findings suggest that the marginal effect of students’ 
comfort level with technology may contribute to their enhanced understanding and retention of concepts when 
presented in this format. 
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CONCLUSION 

This article makes a strong argument for the use of multimedia modules in the form of pre-lectures to complement 
the way in which economics concepts are conveyed to students. Multimedia pre-lectures have been found not only 
to promote retention of information, but also to aid significantly in the understanding of concepts seen for the first 
time by allowing the brain to make visual connections between images and concepts. 

To study the effectiveness of visuals empirically, we used a clinical trial to compare the understanding and retention 
of economics concepts by students placed in a study group that received multimedia pre-lectures of the concepts, 
and a control group that received only written text of the concepts. Students in the study (video) group scored almost 
6 points higher on average on the initial assessment. This difference in scores increased to almost 8 points in a 
second assessment that both groups took 2 weeks after the initial assessment. Therefore, our findings provide strong 
evidence that students can learn and retain concepts more effectively when conveyed using a medium that 
incorporates a greater visual presentation as opposed to just words alone. These findings support those found by 
researchers in other disciplines such as in physics (Stelzer et al., 2009). 

The importance of these findings is highlighted by the fact that more instructors are choosing to flip their 
classrooms. Some instructors create time for active learning by simply replacing traditional lectures (i.e., chalk-and- 
talk) from the student’s learning path with assigned textbook readings before class. Our findings suggest this 
strategy may be counterproductive, because it denies the student an effective way of receiving basic information for 
the first time: the spoken lecture. Instead, we suggest that instructors move the traditional lecture outside the 
classroom by assigning students pre-recorded multimedia lectures that better replicate the chalk-and-talk experience, 
thereby preparing students for a more effective active learning experience in the classroom. 

There are certainly additional avenues of future research. For example, although our results suggest that 
comprehension and retention may improve even further if students are given both mediums (videos and text), 
clinical trials of this effect have not yet been tested. In addition, additional study on whether learning styles affect 
outcomes may be useful, as one could argue that some students are more visual learners while others are not. 
Moreover, it would be useful to determine whether our results apply to advanced courses that are more technical. 

It is important to caution readers, however, that our results do not suggest that instructors should stop using 
textbooks in their courses. A textbook serves many purposes beyond simply introducing students to the material for 
the first time. A textbook serves a critical function as a reference for more in-depth discussion and analysis of the 
concepts, as well as a mechanism to convey and discuss important real-world applications. 

Our results do suggest that textbooks are not as efficient as multimedia videos at introducing concepts to novices for 
the first time. This conclusion has particular relevance to any instructors trying to flip their classroom, because it 
suggests that replacing the typical chalk-and-talk lecture by assigning readings from the textbook results in a less 
efficient tradeoff for the student. A more effective way of flipping the classroom, at least for introductory courses, is 
to offer the chalk-and-talk lecture to students in the form of a multimedia presentation. 

As new resources become increasingly available to prepare students for class, opportunities to improve the learning 
experience by students and the teaching experience by instructors will increase. This is especially the case as more 
instructors adopt active learning exercises in their classes, which encourages students to become teachers, and 
teachers to become facilitators. As online courses increasingly replace chalk-and-talk classes, the flipped classroom 
may be the face-to-face teaching method that ultimately survives. 
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